The present application is concerned with inter-component prediction in multi-component picture coding such as between luma and chroma.
In image and video signal processing, color information is mainly represented in a color space typically consisting of three components like R′G′B′ or Y′CbCr. The first component, Y′ in the case of Y′CbCr, is often referred to as the luma and the remaining two components, the Cb and the Cr components or planes in the case of Y′CbCr, are referred to as the chroma. The advantage of the Y′CbCr color space over the R′G′B′ color space is mainly the residual characteristic of the chroma components, i.e., the chroma components contain less energy or amplitude comparing to the chroma signals of absolute color spaces like R′G′B′. In particular for Y′CbCr, the luma component implies the grey scale information of the image or video and the chroma component Cb implies the difference relative to the blue primary, respectively Cr denotes the difference relative to the red primary.
In the application space of image and video compression and processing, Y′CbCr signals are advantageous as compared to R′G′B′ due to the fact that the color space transformation from R′G′B′ to Y′CbCr reduces or removes the correlation between the different color components or planes. In addition to the correlation removal, less information has to be transmitted, and hence, the color transformation acts as a compression approach too. Such a pre-processing in correlation removal or reduction enables higher compression efficiency while maintaining or increasing the complexity in a meaningful amount as an example. A hybrid video compression scheme is often designed for Y′CbCr input because the correlation between the different color components is removed or reduced and the designs of hybrid compression schemes only have to consider the separate processing of the different components. However, the transformation from R′G′B′ to Y′CbCr and vice versa is not lossless, and hence, information, i.e., sample values available in the original color space might be lost after such a color transformation. This issue can be avoided by using color spaces involving a lossless transformation from the original color space and back to the original color space, e.g., the Y′CoCg color space when having R′G′B′ input. Nevertheless, fixed color space transformations might lead to sub-optimal results depending on the application. For image and video compression, fixed color transformations are often sub-optimal for higher bit rates and non-natural signals with high or without correlation between the color planes. In the second case, a fixed transformation would introduce correlation between the different signals, and in the first case, the fixed transformation might not remove all the correlation between the different signals. Furthermore, due to the global application of the transformation, correlation might not be completely removed from the different components or planes locally or even globally. Another issue introduced by a color space transformation lies in the architecture of an image or video encoder. Usually, the optimization process tries to reduce a cost function, which is often a distance metric defined over the input color space. In the case of transformed input signals, it can be difficult to achieve an optimal result for the original input signal due to additional processing steps. Consequently, the optimization process might result in a minimum cost for the transformed signal but not for the original input signal. Although the transformations are often linear, the cost calculation in the optimization process often involves a signaling overhead and the cost for the final decision is then calculated by a Lagrangian formula. The latter might lead to different cost values and different optimization decision. The color transformation aspect is especially crucial in the domain of color representation as modern image and video displays usually use the R′G′B′ color composition for content representation. Generally speaking, transformations are applied when correlation within the signal or between the signals should be removed or reduced. As a consequence, the color space transformation is a special case of the more generic transformation approach.
Accordingly, it would be favorable to have a multi-component picture coding concept at hand which is even more efficient, i.e. achieves higher bitrates over a broader range of multi-component picture content.